A 3D Head Tracker for an AutolTIatic Lipreading System
نویسندگان
چکیده
A real world automatic lip reading system must be able to cope with movement of the speaker's head during operation. The observed mouth shape depends not only on the true shape of the mouth, but also the angle at which the mouth is viewed. As the speaker's head moves and rotates the viewing angle changes. The resulting distortion can lead to inaccurate mouth measurement and incorrect phoneme recognition. We have developed a system that robustly measures the dimensions of a speaker's mouth whilst the speaker's head is moving and exhibiting rotations of up to 30 degrees away from the camera. Our system tracks the pose of the speaker's head in 3D, detects the mouth by tracking unadorned lip contours and estimates the 3D locations of the upper and lower lip edges and the mouth corners. The system is demonstrated on a person speaking whilst moving his head in 3D, and the mouth height and width are corrected over 9 seconds of 25Hz video footage.
منابع مشابه
A 3D Head Tracker for an Automatic Lipreading System
A real world automatic lip reading system must be able to cope with movement of the speaker’s head during operation. The observed mouth shape depends not only on the true shape of the mouth, but also the angle at which the mouth is viewed. As the speaker’s head moves and rotates the viewing angle changes. The resulting distortion can lead to inaccurate mouth measurement and incorrect phoneme re...
متن کاملAccommodating for 3D Head Movement in Visual Lipreading
In automatic lipreading, the speaker’s head movement can affect the mouth shape appearing in the captured images independently of the true mouth shape. Such distortion can lead to incorrect recognition of visual speech, thus this problem must be dealt with for a practical application. We have developed a system that accomodates the 3D head movement of the speaker in recognising the mouth shape ...
متن کاملThe UWB 3d talking head text-driven system controlled by the SAT method used for the LIPS 2009 challenge
This paper describes the 3D talking head text-driven system controlled by the SAT (Selection of Articulatory Targets) method developed at the University of West Bohemia (UWB) that will be used for participation in the LIPS 2009 challenge. It gives an overview of methods used for visual speech animation, parameterization of a human face and a tongue, and a synthesis method. A 3D animation model ...
متن کاملOn the Dimensionality of Deformable Face Models
Model-based face analysis is a general paradigm with applications that include face recognition, expression recognition, lipreading, head pose estimation, and gaze estimation. A face model is first constructed from a collection of training data, either 2D images or 3D range scans. The face model is then fit to the input image(s) and the model parameters used in whatever the application is. Most...
متن کاملA vision-based head tracker for fish tank virtual reality-VR without head gear
A practical and robust head-position tracking method using computer vision is presented. By combining two simple image processing techniques, this tracker can report the position of the user's head in real time. Whole image processing is performed by software running on normal mid-range workstations. This tracker can support desk top virtual reality (also referred to as \ sh tank VR"), thereby ...
متن کامل